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VEHICLE FRONT-VIEW MONITORING SYSTEM 

BACKGROUND OF THE INVENTION 
The present invention relates to a vehicle front-view 
5 monitoring system having a function of taking fail-safe measures 
against images irregularly monitored due to sunlight 
interfering. 

The vehicle front -view monitoring system having CCD 
cameras has recently attracted much attention. The system 

10 recognizes driving conditions such as a distance to a vehicle 
travelling ahead based on images taken by the cameras and warns 
a driver of danger or controls the vehicle such as by shiftdowns. 

Recognition of the driving conditions using a stereo- 
distance measuring technique obtains a positional difference 

15 (parallax) on an object in a pair of the images and calculates 
the distance to the vehicle travelling ahead by trigonometrical 
survey . 

Such a vehicle front -view monitoring system requires 
fail-safe measures for securing safety driving. A failure to be 
20 detected is a fault monitoring condition such that the vehicle 
in front is beyond recognition while monitoring cameras are 
facing weak sunlight. 

Image monitoring failures are discussed with reference to 
FIGS. 1 to 3. 

25 When monitoring vehicles traveling ahead (FIG. 1) while 

facing weak sunlight^ disc-like diffusion could appear on the 
monitored image, as illustrated on the left-upper side in FIG. 
2f due to saturation of luminance. Such a disc-like diffusion 
appearing on the monitored image while the vehicle is facing the 

30 sunlight is called a sunlight -diffusion disc hereinafter. These 
sunlight-diffusion discs cause the saturation of pixel Ixominance 
on the disc areas and the surrounding areas . A correct image data 
therefore can not be obtained, or vehicles in front are masked 
by the sunlight-diffusion discs, which results in monitoring 

35 failures. Moreover, the disc edges have a big luminance change 
between adjacent pixels in the horizontal direction and hence 
incorrect distance data could be calculated on the edges. 
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FIG. 3 illustrates distance-data calculation based on 
stereo images ( a reference ijnage and an image to be compared such 
as shown in FIG. 1) . 

In FIG. 3f black sections have a big luminance change 
5 between the adjacent pixels in the horizontal direction. These 
sections carry distance data (depth). The intensities of most 
pixels in the sunlight-diffusion disc area have been saturated, 
so that no distance data is calculated in this area (a lost state) . 
Moreover, the disc edges have a big luminance change in the 
10 horizontal direction, so that the incorrect distance data could 
be calculated. 

Securing the safety driving in such a fault condition due 
to a generation of the sunlight-diffusion discs discussed above 
requires the fail-safe measures to temporarily halt the 
15 monitoring function. 

SUMMZ^Y OF THE INVENTION 

A purpose of the present invention is to provide a vehicle 
front- view monitoring system for monitoring optical 
20 irregularities occurring on monitored images and swiftly taking 
fail-safe measures. 

The present invention provides the vehicle front -view 
monitoring system for taking the fail-safe measures comprising: 
a camera device for taking an image of a view in front; a calculator 
for calculating luminance data on the image; and a determination 
section for determining whether there is a fail occurring on the 
monitoring system based on the luminance data, the fail-safe 
measures being taken if the fail is occurring. 

BRIEF DESCRIPTION OF DRAWINGS 
FIG. 1 illustrates a normal image monitored on a monitor 
25 screen; 

FIG. 2 illustrates an irregular image monitored on the 
monitor screen on which sunlight-diffusion discs are generated; 

FIG. 3 illustrates the irregular image monitored on the 
monitor screen, including inaccurate distance data due to 
30 generation of sunlight-diffusion discs; 
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FIG* 4 is a block diagram of a preferred embodiment of the 
vehicle front -view monitoring system according to the present 
invention; 

FIG, 5 shows a flow chart of a weak-sunlight-facing driving 
5 determination routine; 

FIG. 6 shows a flow chart of a weak-sunlight- facing driving 
determination routine; 

FIG. 7 illustrates the upper and lower areas set on the 
monitor screen; 

10 FIG. 8 illustrates an area on which a vehicle travelling 

ahead is monitored; 

FIG. 9 illustrates a monitoring area; 

FIG. 10 illustrates measured luminance-distribution 
characteristic parameters ; 
15 FIG. 11 illustrates normalized luminance-distribution 

characteristic parameters ; 

FIGS. 12A and 12B illustrate luminance addition 
characteristics in normal driving; 

FIGS. 13A and 13B illustrate luminance addition 
20 characteristics in weak-sunlight- facing driving; 

FIGS. 14A to 14C illustrate luminance center; 

FIG. 15 shows a flow chart of a fail determination routine; 

FIG. 16 shows a flow chart of a count-up amount setting 
routine; 

25 

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS 
Preferred embodiments according to the present invention 
will be disclosed with reference to the attached drawings. 

FIG. 1 shows a block diagram of a preferred embodiment of 
30 a vehicle front -view monitoring system according to the present 
invention. 

A pair of cameras 1 and 2 having image sensors such as CCDs 
are attached to a vehicle such as an automobile in the vicinity 
of tyres with a predetermined distance (corresponding to 
35 camera-baseline length) in the direction of vehicle width, for 
taking images ahead. 

The main camera 1 (right camera) takes a reference image 
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required for stereo processing whereas the sub-camera 2 (left 
camera) takes an image to be compared. 

Analog images taken by the cameras 1 and 2 while the main 
camera 1 and sub-camera 2 are synchronized with each other are 

5 converted into a specific luminance gradation (for example, gray 
scale at 256 gradations). 

The digitized images are processed by an linage correcting 
section 5 for correction of luminance, image geometrical 
conversion, etc. An inevitable misalignment on the cameras 1 and 

10 2 will affect left and right images . Such effect can be cancelled 
by geometrical conversion such as an image rotation and a parallel 
shift with Affine conversion, etc. Luminance data on the stereo 
linages (a reference image and an image to be compared) for one 
frame corrected by the image correcting section 5 are stored in 

15 an image memory 8. 

The luminance data are further supplied to a stereo -image 
processor 6 for calculating distance data (parallax for each 
object). Each distance data corresponds to a positional 
deviation on the same object in the stereo images. An actual 

20 distance to the object is defined on the basis of parallax by 
a trigonometrical survey. The distance data for one frame thus 
obtained are stored in a distance-data memory 7 . 

The luminance data and the distance data are supplied to 
a microcomputer 9. 

25 The microcomputer 9 includes a road-recognizing section 

10, a three-dimensional (3-D) object-recognizing section 11, a 
fail-determining section 12 and a processor 13. 

In response to the luminance and distance data, the 
road-recognizing section 10 recognizes the condition of a road 

30 ahead, and the 3-D object-recognizing section 11 recognizes 
three-dimensional objects (vehicles travelling ahead). 

Based on the data from the sections 10 and 11 , the processor 

13 warns the driver of a danger via a warning device 19 such as 
a display or a speaker when it determines that the warning is 

35 necessary. 

Moreover, the processor 13 controls several controllers 

14 to 18 if necessary. For instance, it sends a command signal 
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to an automatic transmission (AT) controller 14 for downshifts 
or to an engine controller 18 for an engine power-down. The 
processor 13 may further sends the command signals to an 
antilock-braking system (ABS) controller 15, a traction control 
5 system (TCS) controller 16 and a vehicle controller 17 for 
controlling an engine speed and a torque distribution to tyres, 
and etc. 

The fail-determining section 12 determines whether the 
cameras 1 and 2 are taking the images into the sun and a fail 

10 is occurring, based on the luminance and the distance data stored 
in the image memory 8 . The vehicle and the warning controls are 
interrupted by the fail-safe measures for prevention of 
malfunction which could otherwise occur due to lowering in 
monitoring accuracy during a period of determination in that a 

15 fail is occurring, or of "1" set for a fail flag NG which will 
be described later. 

FIGS. 5 and 6 show flowcharts for indicating a routine of 
determining whether the cameras 1 and 2 are taking images into 
the sun while the vehicle is facing the sunlight, which is called 

20 a sunlight- facing driving hereinafter. The routine is repeated 
by the fail-determining section 12 by a predetermined interval 
(a cycle of 100 ms for example). 

The requirements for taking the fail-safe measures (called 
fail-safe measure-interruption requirements hereinafter) are 

25 discussed before explaining the routine in detail. 

Requirements for steps SI to S6 are necessary for taking 
the fail-safe measures in the present invention. It is 
determined that the cameras 1 and 2 are taking the images into 
the sun only when all the fail-safe measure-interruption 

30 requirements are met in steps SI to 86 and the routine proceeds 
to step S12 whereas to step S7 if not met in any one of steps 
SI to S6. 

As disclosed later, the fail determining section 12 
determines whether the fail-safe measure- interruption 
35 requirement or a fail-safe measure-release requirement is met 
and performs counter increment or decrement in accordance with 
a result of determination. The fail-safe measure-release 
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requirements are the requirements for resuming functions that 
have been interrupted by taking the fail-safe measures. 

The interruption requirements are required for 
determining whether the cameras 1 and 2 are taking images into 
5 the sun, so that the fail-safe measures should be taken. A 
counter increment is performed when an interruption requirement 
is met. The fail-safe measures will never be taken before the 
counter reaches a predetermined value, even if the interruption 
requirements are met. 
10 The release requirements are required for determining 

whether to halt the counter increment . In the present invention, 
the counter starts a count-up when an interruption requirement 
is met and continues until a release requirement is met even though 
monitoring situation is changing and no interruption requirement 
15 is met anymore. 

The present invention deploys the following six 
interruption requirements for taking fail-safe measures, however, 
any combination of at least two of the requirements or addition 
of other requirements are available. 
20 First Requirement: No vehicle is recognized ahead, or no 

distance to a vehicle in front is calculated (step SI). 

The fail-safe measures will be taken in the present 
invention when the cameras 1 and 2 are taking the images into 
the sun (sunlight- facing driving condition) so that the vehicle 
25 that actually traveling ahead is lost on a monitor screen of the 
monitoring system. In other words, the fail-safe measures will 
not be taken even though the sunlight-diffusion discs shown in 
FIG. 2 are appearing while the vehicle traveling ahead is 
recognized. 

30 The present invention therefore deploys a decision in that 

no vehicles are recognized ahead as a fail-safe measure- 
interruption requirement. This decision step is executed even 
though no vehicles actually exist ahead. 

When the requirement in step SI (FIG. 5) is met, the routine 

35 proceeds to step S2 to determine whether the cameras 1 and 2 are 
taking the images into the sun (sunlight-facing driving 
condition) . 
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Calculation of the distance to the vehicle in front in step 
Si is explained before disclosing the step S2, 

The distance to the vehicle in front is calculated in 
combination of a stereo-distance measuring technique and an 
5 ijnage-recognition technique. 

Calculated first is a lane model that indicates a 
three-dimensional shape of a road ahead. Lane markings painted 
on the left and right sides of a lane and displayed on the monitor 
screen have the luminance higher than that of the road surface. 
10 The markings are extracted as areas having luminance edges on 
the borders between that and the road surface. 

The parallax on the lane thus extracted is calculated for 
each (4 x 4) -pixel block, for example. A two-dimensional 
position and the parallax in the monitored image for one pixel 
15 block in which the lane is displayed provide a three-dimensional 
position and a distance of the lane portion by using a known 
coordinate conversion formulas. 

The calculated three-dimensional lane distribution is 
approximated into a straight line for a predetermined segment. 
20 The straight lines are connected into a polygonal line for 
calculating the lane model. 

The vehicle travelling ahead is recognized as a three- 
dimensional object existing on the area (road surface) surrounded 
by left and right lane models. The distance to the vehicle in 
25 front is calculated based on the parallax of the object. 

These techniques are disclosed in detail in Japanese 
Patent Application No. 11-269578 (road recognition) and Japanese 
Unexamined Patent Publication No. 5-265547 (recognition of 
vehicle traveling ahead) . 
30 Second Requirement: A shutter speed "a" is equal to or lower 

than a predetermined reference speed (step S2). 

The shutter speed tends to be higher in a sunlight-facing 
driving condition in that the vehicle in front is missing on the 
monitor screen than that in a usual driving condition. In view 
35 of this tendency, an appropriate reference shutter speed such 
as 2000 usee is set. 

The present invention deploys a decision in that the 



8 



shutter speed "a" is lower than this reference shutter speed as 
a fail-safe measure- interrupt ion requirement. 

The shutter speed, however, tends to be extremely higher 
against a strong sunlight in mid-summer, for example 200 jusec. 
5 or lower. In view of this tendency, the routine proceeds to step 
S3 (FIG. 5) when the requirements in steps 51 and S2 are both 
met. 

Third Requirement: The upper luminance saturation factor 
LRl is larger than the lower luminance saturation factor LR2 ( step 
10 S3). 

Sunlight-diffusion discs images shown in FIG. 2 will be 
generated on the upper side of the monitor screen when a vehicle 
in front is missing during sunlight-facing driving. This 
phenomenon has a tendency in that the luminance saturates on the 
15 upper side. 

In view of this tendency, as illustrated in FIG. 7, the 
upper and the lower areas are provided in a reference image. 

The present invention deploys the decision in that the 
upper liminance saturation factor LRl in the upper area is larger 
20 than that of the lower luminance saturation factor LR2 in the 
lower area as a fail-safe measure- interruption requirement. 

The luminance saturation factors LRl and LR2 correspond 
to a ratio of the number of pixels for which luminance has 
saturated (for example, the luminance 240 or more) to the total 
25 number of pixels in the upper and the lower areas, respectively. 

The requirement in step S3 could be met even though no 
sunlight-diffusion discs (FIG. 2) are generated when there is 
no shielding in a fair day or while travelling on a bridge. In 
view of this, the routine proceeds to step S4 (FIG. 5) when the 
30 requirements in steps SI, S2 and S3 are all met. 

No reference values are set for the upper and lower 
l"uminance saturation factors LRl and LR2 in this embodiment. 
This is because factors tend to vary very largely according to 
the driving conditions . Particularly, the lower factor LR2 often 
35 becomes zero in a normal driving condition. 

Fourth Requirement: The number of luminance edges EN for 
an area of the vehicle in front is smaller than a predetermined 
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reference number (step S4). 

While the vehicle travelling ahead is missing on the 
monitor screen during the sunlight- facing driving, the number 
of the luminance edges EN on the sunlight-diffusion disc area 

5 {FIG. 2) tends to become extremely small • 

The number of the luminance edges EN indicates the number 
of a pair of adjacent pixels in the horizontal direction on the 
monitor screen for which variation in luminance is more than a 
predetermined reference value - 

10 As illustrated in FIG. 8, a vehicle in front area is set 

in the upper-mid section of the monitor screen, in which the 
vehicle travelling ahead would be displayed. 

The present invention deploys the decision in that the 
nttmber of the luminance edges EN in the vehicle in front area 

15 (FIG* 5) is smaller than an appropriate reference number as a 
fail-safe measure- interruption requirement. 

This requirement could, however, be met in driving at night, 
in tunnel, mist, smoke or on a snowy road. In view of this, the 
routine proceeds to step S5 (FIG. 5) when the requirements in 

20 steps SI, S2, S3 and S4 are all met. 

The number of the calculation of the distance data in the 
vehicle in front area can be used instead of the number of the 
luminance edges EN. This is because the number of the luminance 
edges EN and the number of the calculation of the distance data 

25 have a strong correlation in a stereo-matching, or the lager the 
nxamber of the luminance edges EN, the larger the number of the 
calculation of the distance data. 

Fifth Requirement: The luminance-distribution- 
characteristics parameter normalized by using a shutter speed 

30 "a" meets a requirement. 

As shown in FIG. 9, several slit-like monitoring areas Ni 
are arranged on the upper side of the monitor screen in the 
horizontal direction. The reason for the arrangement of 
monitoring areas Ni on the upper side is that the luminance mostly 

35 varies on the upper side when the vehicle is facing a weak 
sunlight. 

The monitoring areas Ni are provided for evaluating the 
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luminance-distribution characteristics for a monitored image in 
the horizontal direction. 

For example, a reference image is divided into blocks of 
16 pixels each in the horizontal direction when the reference 
5 iruage is composed of 200 and 512 pixels in the vertical and 
horizontal directions, respectively. Therefore, 32 monitoring 
areas Ni of (200 x 16) pixels are set. 

Calculated next is addition of luminance Ai for each 
monitoring area Ni. The addition of liiminance Ai for a monitoring 
10 area Ni can be calculated by adding the luminance of sample pixels 
that are uniformly distributed over the monitoring area Ni. The 
added luminance of the sample pixels may further be averaged. 

The addition of luminance Ai is the value, for example, 
obtained by adding the luminance of 4 pixels sampled in the 
15 horizontal and vertical directions each. Pixels uniformly 
distributed over the monitoring area Ni can be used for 
calculating the luminance over the area Ni with less amount of 
calculation. The addition of luminance Ai can be calculated for 
all pixels involved in the area Ni if the amount of calculation 
20 is out of question. 

The Ai-distribution characteristics for the 32 monitoring 
areas corresponds to the image distribution characteristics in 
the horizontal (lateral) direction. 

The parameter (Ixaminance-distribution characteristic 
25 value) that indicates luminance-distribution characteristics on 
the addition of luminance Ai is normalized by a shutter speed 
"a". Used here as the parameter is luminance-addition variance 
VAR that indicates a variance in the addition of luminance Ai. 
Limiinance-addition variance VAR' normalized by the shutter speed 
30 "a", called a normalized luminance-addition variance VAR*, is 
calculated by the following equations: 
VAR' = VAR/a 

VAR^ = {(AI - ' + (A2 - Kv.) ' + - (An - A,,J '}/n = Z 

(Ai')/n -( ZAi/n) ' (1) 
35 The normalized luminance-addition variance VAR' as the 

luminance-distribution-characteristics parameter requires a 
decision in that the VAR' is larger than an appropriate reference 
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value as of fail-safe measure- interruption requirement. 

Instead of the VAR' ^ the maximum addition of luminance Amax 
that indicates the maximum (peak) of the addition of luminance 
Ai can be used as the luminance-distribution-characteristics 
5 parameter . 

The Amax requires a decision in that the maximum addition 
of luminance Amax ' normalized by the shutter speed "a", called 
normalized maximum addition of luminance Amax ' , is larger than 
an appropriate reference value as a fail-safe measure- 
10 interruption requirement. 

Another decision in that both the normalized VAR' and the 
normalized Amax' are larger than the corresponding appropriate 
reference values can be used as a fail-safe measure-interruption 
requirement . 

15 The reason for using the normalized VAR' and Amax ' for 

determination of whether the vehicle is facing the weak sunlight 
is discussed with reference to FIGS. 10 and 11. 

FIG. 7 shows the un-normalized characteristics of the 
luminance-distribution-characteristics parameters ( VAR and 

20 Amax) that were measured in an actual driving test. Images were 
taken into the weak sunlight from about 300 sec. to about 340 
sec. after the start of measurement. 

The measurement of Amax teaches that Amax was measured in 
a fixed range through the test and hence it is difficult to find 

25 out an extreme tendency on a weak-sunlight-facing driving. 

The measurement of VAR shows a peak on about 270 sec. after 
the start of measurement, which appeared while the vehicle was 
not facing the sun, and hence it is also difficult to find out 
an extreme tendency on the weak-sunlight-facing driving. 

30 FIG. 11 shows the normalized characteristics of the 

Iviminance-distribution characteristics parameters (VAR' and 
Amax ' ) . 

Compared to the test results shown in FIG. 10, the results 
in FIG. 11 show an extreme tendency in which both VAR' and Amax ' 
35 are extremely increased while the vehicle is facing the weak 
sunlight. 

It is therefore apparent that these parameters VAR' and 
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Amax' can be used effectively for determining whether the vehicle 
is facing the weak sunlight. 

Accordingly, the present invention employs the parameters 
Amax ' and VAR' that are obtained by normalizing the parameters 
5 Amax and VAR with a shutter speed "a" for determination of the 
weak sunlight-facing driving in view of the fact that Amax and 
VAR are the functions of the shutter speed "a" . 

The decision in that both Amax ' and VAR' are larger than 
the corresponding reference values as a fail-safe measure- 
10 interruption requirement further enhances accuracy of 
determination on fail-safe measurements. 

The shutter speed, however, tends to be higher in a fair 
day, and hence the routine proceeds to step S6 (FIG. 5) when the 
requirements in steps SI, S2, S3, S4 and 85 are all met. 
15 Six requirement: Division of the liaminance moment by the 

total addition of the luminance is equal to or lower than a 
reference value (step S6). 

Step S6 evaluates the distribution of addition of 
luminance Ai for each monitoring area Ni from the point of view 
20 which is different from step S5. 

FIGS. 9A and 9B show the distribution characteristics of 
the addition of luminance Ai in regular conditions. 

In detail, FIG. 9A shows an example of the distribution 
characteristics for a driving condition such that a vehicle is 
25 entering a freeway, so that the sky is displayed all over the 
upper section of the monitor screen. 

FIG. 9B shows another example of the distribution 
characteristics for a driving condition such that the vehicle 
is travelling on a road uncovered with snow, which is displayed 
30 in the mid section of the monitoring screen, while the ground 
besides the road has been covered with snow. 

On the other hand, FIGS. lOA and lOB show the distribution 
characteristics of the addition of luminance Ai while the cameras 
1 and 2 is facing a weak sunlight during the sunlight-facing 
35 driving . 

The weak-sunlight-facing driving results in a tendency in 
that high luminance portions being converged on one section 
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corresponding to the sunlight-diffusion disc (FIG. 2) of a 
monitored image in the horizontal direction* This is shown in 
FIG. 13A in which the high luminance portions are converged on 
the mid section of the monitor screen. FIG- 13B shows high 
5 luminance portions converged on the right section of the monitor 
screen • 

Step S6 determines whether the present driving condition 
is a regular one such as shown in FIG. 12A or 12B or the 
weak-sunlight-facing driving such as shown in FIG. 13A or 13B. 
10 For this determination^ a luminance center LC and a 

liiminance moment LM are introduced as defined in the following 
equations : 

LC = I (i X Ai)/lAi 
LM = Z ( I i - LC i X Ai) (2) 
15 where "i" indicates a monitoring area Ni- number (0 to 31) 

and Ai indicates an addition of the luminance Ai in the i-th 
monitoring area Ni, like in step S5. 

The luminance center LC is defined as a horizontal position 
at which the addition of luminance Ai in the horizontal direction 
20 are converged. The LC is obtained by dividing the total of the 
product of Ai x " i" (=horizontal position) by the total of Ai. 

For example, when all of the addition of luminance Ai are 
the same value as shown in FIGF. 14A, the Ixaminance center LC 
is 15. 5 that lies between 0 and 31. Moreover, when the addition 
25 of luminance AO to A15 are all zero and A16 to A31 are the same 
value (9^: 0), the luminance center LC is 23. 5. 

The luminance moment LM is defined as the total of the 
product of a distance from the luminance center LC to a horizontal 
position "i" times the addition of luminance Ai. The moment LM 
30 indicates the distribution of the addition of luminance Ai in 
the horizontal direction on the basis of the luminance center 
LC. The luminance moment LM becomes small when the addition of 
luminance Ai is small in overall or the luminance is converged 
in the vicinity of the luminance center LC. 
35 Calculated next is the value obtained by dividing the 

luminance moment LC by the total of addition of luminance 

lAi, or liiminance moment LM' that is normalized by ZAi 
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as indicated by the following equation: 
LM' = LM/lAi 

= Z ( 1 i - LC I X Ai) lAi ... (3) 
In normal driving at the distribution characteristics such 
5 as shown in FIG. 14A, the luminance moment LM tends to become 
large because luminance are distributed all over in the 
horizontal direction. 

On the other hand, in the sunlight- facing driving at the 
distribution characteristics such as shown in FIG. 14C, the 
10 luminance moment LM tends to become small because the luminances 
are converged in the vicinity of the luminance center LC. 

However, it is sometiiaes difficult to discriminate the 
sunlight-facing driving condition from a normal driving 
condition because the luminance moment LM becomes large for a 
15 bright image even in sunlight-facing driving. 

In view of this, the normalized luminance moment LM' that 
is obtained by dividing the luminance moment LM by the total of 
addition of luminance Ai and that depends only on luminance 
distribution is used for the determination in step 36. 
20 The normalized luminance moment LM' has a tendency of 

becoming large for the distribution characteristics shown in FIGS • 
12A and 12B whereas small in FIGS. 13A and 13B. 

In view of this, step S6 compares the normalized luminance 
moment the LM' with an appropriate reference value for detecting 
25 the weak-sunlight-facing driving condition that is shown in FIGS . 
13A and 13B. 

The equation (3 ) can be modified for a large weight of the 
addition of luminance Ai for evaluation of Ai distribution as 
follows : 

30 LM' = Z ( I i - LC I X Ai^)/ lAi 

LC =Z(i X - Ai')/ ZAi ... (4) 

In FIG. 5, when all the requirements in steps SI to 86 are 
met, or all the fail-safe measure-interruption requirements are 
met, it is determined that the vehicle is facing a weak sunlight. 
35 A count flag Font is then set at "1" (step S12) . The count flag 
Fcnt is set at "0" when a counter K reaches the lower limit value 
0 whereas "1" while the counter K is counting up or down (K ^ 
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0). 

The counter K is set at a count value a (> 0) in the 
following step S13. 

The counter K is used for a fail determining routine such 
5 as shown in FIG. 15 for determining whether to take fail-safe 
measures • 

When the counter K reaches the upper limit value Kllm (step 
S30) due to increment in the weak- sunlight -facing driving 
condition determining routine^ a fail flag NG is set at "1" (step 
10 S31). On the other hand, when the counter K reaches the lower 
limit value 0 (step S32), the count flag Font and the fail flag 
NG are both reset at "0" (steps S33 and S34). 

Moreover^ when the counter K is in the range from the lower 
limit value 0 to the upper limit value Klim, the routine ends 
15 via steps S30 and S3 2 with no change in the count flag Font and 
the fail flag NG, 

While a front-view monitoring is interrupted by the 
fail-safe measures (NG =1), it will not be resumed (fail-safe 
measures will not be released) unless the counter K reaches the 
20 lower limit value 0. 

On the other hand, while the front-view monitoring is 
performed (NG = 0), it will not be interrupted by the fail-safe 
measures unless the counter K reaches the upper lower limit value 
Klim > 

25 The counter K starts the count-up when a fail-safe 

measure-interruption requirement is met. The fail flag NG is set 
at "1" when such a driving condition continues for a predetermined 
period (decided based on the upper limit value Klim and the 
count-up amount a ) for taking the fail-safe measures. 
30 Monitoring is thus interrupted for avoiding a decrease in 
monitoring control accuracy in the weak-sunlight-facing driving. 

The count -up amount or is decided according to a count- 
up amount setting routine shown in FIG. 6. The amount a is 
decided in accordance with a distance variable and a difference 
35 in luminance in left and right cameras (step S20). 

Usually, cameras are facing the sunlight in the 
sunlight- facing driving and hence the variation in calculation 



16 



of the distance tends to be larger compared to a normal driving 
condition. 

In view of such a tendency, the present invention obtains 
the distance variable in a predetermined area in a monitored image, 
5 for example, an area extending in the vertical direction in the 
image. The count-up amount a is set at a large one when the 
variable is large. It is also set at a large amount in the 
sunlight-facing driving due to a larger luminance difference 
between the left and right cameras compared to the normal driving. 
10 On the other hand, it is determined that the vehicle (the 

cameras 1 and 2) is not facing the sunlight if any one (or more) 
of the fail-safe measure- interruption requirements is (or are) 
not met through steps SI to S 6 in FIG. 5. 

The routine then proceeds to step S3 in FIG. 6 to determine 
15 whether the count flag Fcnt has been set at "1", or the counter 
K has been counting up or down in step S7. If it is determined 
as K = 0, the routines will not proceed anymore and ends. 

As disclosed, monitoring is continued without taking any 
fail-safe measurements during the normal driving condition 
20 without facing sunlight and the count K having reached the lower 
limit value 0, or the fail flag NG having been set at "0". 

On the contrary, when any one (or more) of the fail-safe 
measure- interrupt ion requirements is (or are) not met through 
steps 81 to S6, but it is " YES'' (Fcnt=l, K 0) in step S7, 
25 the routine proceeds to step 58. 

Steps S8 to SIO define fail-safe measure-release 
requirements for the counter K to count up or down to resume 
functions that have been interrupted by the fail-safe measures . 
The counter K will count down when a fail-safe measure-release 
30 requirement is met whereas count up when a requirement is not 
met. 

The fail-safe measure-release requirements are not so 
severe than the fail-safe measure-interruption requirements for 
avoiding repetition of the fail-safe measures while it is 
35 repeated that the vehicle travelling ahead is lost and appeared 
on the monitoring screen. 

As one of the fail-safe measure-release requirements, it 
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is determined in step S8 whether the shutter speed "a" is 3000 
usee, or lower. If not, or this fail-safe measure-release 
requirement is met, the routine proceeds to step Sll in which 
the counter K counts down by a count-down amount 0 • 

5 On the other hand, if " YES" in step S8, the routine 

proceeds to step S9 to determine whether the luminance-addition 
variance VAR is equal to or larger than the reference value. 

The VTR tends to be large in the sunlight- facing driving 
condition because, particularly, when facing a strong sunlight, 

10 luminance-saturated sections appear on the monitor screen due 
to smears or sunlight-diffusion discs (FIG. 2) and also blackout 
sections. 

However, VTR tends to be large not only in the 
sunlight-facing driving, but also, for instance, in a sunny day, 
15 due to a strong contrast between sunny and shade places . It tends 
to be large, particularly, there is the shade of guard rails or 
roadside trees along the road. 

In view of such a tendency, the present invention deploys 
the luminance-addition variance VTR that will not vary so much 
20 in the weak-sunlight-facing driving compared to the normalized 
luminance-addition variance VTR' , for the fail-safe measure- 
release requirements less severer than the fail-safe 
measure- interruption requirements, to avoid the repetition of 
fail-safe measures discussed above. 
25 If " YES'' in step S9 (FIG. 6) , the routine proceeds to step 

513 (FIG. 5 ) in which the counter K starts count-up. On the other 
hand, if "NO" in step S9, the routine proceeds to step SIO to 
detennine whether lane markings have been accurately recognized. 
Accuracy on a lane marking recognition can be used as a fail-safe 
30 measure-release requirement for avoiding the repetition of the 
fail-safe measures discussed above. This is because the road 
surface will be mostly correctly displayed on the monitor screen 
and hence the lane marking recognition accuracy will not be 
lowered so much even in the weak- sunlight- facing driving. 
35 If the lane marking recognition accuracy has been lowered 

in step SIO, the routine proceeds to step S13 (FIG. 5) in which 
the counter K starts count-up. 
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On the other hand, if the lane marking recognition accuracy 
is not lowered in step SlO, the routine proceeds to step Sll in 
which the counter K starts count-down. 

The lane marking recognition accuracy can be determined 

5 as disclosed in Japanese Patent Application No, 11-269578 • The 
accuracy of lane markings detected on a monitored image is 
determined based on the number of edges of luminance on the lane 
markings, the positional relationship (continuity) to the lane 
markings detected previously and parallelism between left and 

10 right lane markings. High lane marking recognition accuracy is 
achieved as the number of edges of luminance is larger, the lane 
markings detected in the present image more continues to the 
previously detected markings and the detected left and right 
markings are more parallel to each other. 

15 As clearly understood from the foregoing disclosure, the 

fail-safe measure-release requirements (count-down 
requirements for the counter K) are as follows: 

(1) the shutter speed "a" is higher than 3000 juse., or 

(2) if not, the luminance-addition variance VAR is smaller 
20 than the reference value and lane marking recognition is 

accurate. 

As disclosed, the present invention monitors parameters 
such as the luminance-distribution characteristics parameter, 
the number of edges of luminance, the upper and lower luminance 

25 saturation factors and the luminance moment for detection of a 
weak-sunlight-facing driving condition. 

The fail-safe measures can therefore be taken exactly 
during the weak- sunlight- facing driving, thus offering a highly 
reliable vehicle front-view monitoring system. 

30 Particularly, in the present invention, several 

monitoring areas are set on the monitor screen, which are 
slit-like areas extending vertically on a monitored image for 
obtaining the luminance-distribution characteristic values (the 
luminance-addition variance VAR and the maximum addition of 

35 luminance Amax) in the horizontal direction of the image. 

The luminance-distribution characteristic values are 
normalized using a shutter speed "a" for obtaining parameters 
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(the liiminance-addition variance VAR' and the normalized maximvim 
addition of luminance Amax» ) that are highly related to 
weak-sunlight-facing driving. 

The normalized parameters VAR* and Amax* can be used for 
detection of a weak-sunlight-facing condition, which is more 
accurate than using the un-normalized parameters VAR and Amax 
as the fail-safe measure-interruption requirements. 

The fail-safe measure-release requirements are set as 
including several conditions in addition to a weak- sunlight- 
facing driving condition so that it is not so severe than those 
of the fail-safe measure-interruption requirements. 

This requirement setting avoids the repetition of 
unnecessary fail-safe measures such as the warning which could 
irritate the driver while it is repeated that the vehicle 
travelling ahead is lost and appeared on the monitoring screen. 

The fail-safe measure- interruption requirements include 
the normalized Luminance-addition characteristic values VAR' and 
Amax' whereas the fail-safe measure-release requirements include 
VAR and Amax , the parameters different from VAR' and Amax because 
those are not normalized. 

Moreover, the fail-safe measure-release requirements 
include the accuracy on the lane marking recognition that is not 
deployed for the fail-safe measure- interruption requirements. 

Deploying different parameters for the fail-safe 
measure-interruption/release requirements offers flexible 
fail-safe requirement settings that match an actual driving 
condition compared to setting in which reference values are 
changed between the interruption/release requirements for the 
same parameters. 

The count-up amount a is adjusted based on the luminance 
difference between the left and right cameras and distance data 
variable. 

This adjustment offers variable settings for a fail-safe 
determination time from a moment at which the fail-safe 
measure- interruption requirements are met to a moment of taking 
fail-safe measures in accordance with actual driving conditions . 

Disclosed so far is the vehicle front-view monitoring 
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system that takes fail-safe measures against a weak- 
sunlight-driving condition in which the vehicle is travelling 
as facing the weak sunlight so that sunlight-diffusion discs 
appear on the monitor screen. 

However, the present invention is applicable to not only 
such a weak-sunlight-driving condition but also another optical 
phenomenon such as smear occurring on a monitoring screen. 

As disclosed, according to the present invention, optical 
irregularities occurring on the monitored images due to optical 
interfering can be accurately detected, thus, the fail-safe 
measures can be taken swiftly in response to such an optical 
phenomenon for securing the safety driving. 

It is further understood by those skilled in the art that 
the foregoing description is a preferred embodiment of the 
disclosed device and that various change and modification may 
be made in the invention without departing from the spirit and 
scope thereof. 



